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Abstract — Known sparsity thresholds for basis pursuit to de- 
liver the maximally sparse solution of the compressed sensing re- 
covery problem typically depend on the dictionary's coherence. 
While the coherence is easy to compute, it can lead to rather pes- 
simistic thresholds as it captures only limited information about 
the dictionary. In this paper, we show that viewing the dictionary 
as the concatenation of two general sub -dictionaries leads to prov- 
ably better sparsity thresholds — that are explicit in the coherence 
parameters of the dictionary and of the individual sub-dictionaries. 
Equivalently, our results can be interpreted as sparsity thresholds 
for dictionaries that are unions of two general (i.e., not necessarily 
orthonormal) sub-dictionaries. 

I. Introduction 

A typical problem in compressed sensing (CS) [1], [2] is to 
find the sparsest representation of a given vector as a linear 
combination of vectors from a given set, commonly referred to 
as dictionary [3]-[9]. More concretely, let a dictionary be a set 



of TV elements d; <E 



(i = 1, . . . , N), which we assume 



(throughout the paper) to have unit £2 -norm and to span C . 
We organize the dictionary elements into an M x N matrix 
D = [di . . . djv]; typically M < N. Given the vector y € C M , 
the problem of finding its sparsest representation in D can be 
stated as follows: 

(P0) find arg min||x|| subject to y = Dx. 

Here, ||x||o denotes the number of nonzero entries of the vector 
x. Due to the combinatorial nature of the problem, solving (P0) 
is infeasible for practically relevant problem sizes N, M. A 
standard approach in the CS literature is to relax (P0) to the 
following optimization problem, commonly referred to as basis 
pursuit [4]-[9] 

(PI) find argmin||x||i subject to y = Dx 

where ||x||i denotes the ^i-norm of x, and to ask under which 
conditions the solutions of (PI) and (P0) coincide. The advantage 
of this approach resides in the fact that (PI) can be cast into a 
linear program [4] and can therefore be solved (more) efficiently. 

It turns out that the solutions of (P0) and (PI) are unique 
if there exists a vector x that satisfies y = Dx with ||x||o 
less than a certain sparsity threshold [5]-[9] (in the following 
often referred to simply as threshold). This sparsity threshold 
is a function of the dictionary's coherence [5], defined as the 
maximum of the absolute value of the inner product between 
pairs of distinct columns of D, where the maximum is taken 
over all such pairs in the dictionary. It was furthermore shown 
in [8] that a sparsity threshold that guarantees the uniqueness of 



the solution of (PI) necessarily guarantees that this solution is 
also the unique solution of (P0). 

The coherence is easy to compute but often leads to rather 
pessimistic thresholds as it captures only limited information 
about the dictionary. Using additional structural information 
about the dictionary can lead to improved (i.e., higher) thresholds. 
For example, in [7], [10] sparsity thresholds for a dictionary 
that consists of the union of two orthonormal bases (ONBs) are 
derived that are about a factor of 2 higher than the corresponding 
thresholds for a general dictionary with the same coherence. 

Contributions: We consider dictionaries — with coherence 
d — that consist of two general sub-dictionaries with coherence 
a and 6, respectively. The two sub-dictionaries need not be 
ONBs, need not have the same number of elements, and do 
not need to span C M . We denote the set of all such dictionaries 
as T>(d, a, b) ^]the individual dictionaries in T>(d, a, b) need not 
have the same cardinality. 

For dictionaries in T)(d, a, b), we provide a sparsity threshold 
that guarantees uniqueness of the solution of (P0), and a threshold 
that guarantees uniqueness of the solution of (PI) and, hence, 
equivalence of the solutions of (PI) and (P0). These thresholds 
depend on the parameters d, a, and b only and improve on the 
threshold that would be obtained if one treated the union of two 
sub-dictionaries as a general dictionary with coherence d and 
applied the threshold in [6], [8]. The improvement is substantial 
whenever a, b <C d. The thresholds found in [7], [10] for 
the two-ONB setting are special cases of our thresholds. We 
furthermore show that thinking of a general dictionary as the 
concatenation of two (general) sub-dictionaries, the thresholds 
derived in this paper strictly improve upon the one reported in [6], 
[8], unless b = d in which case our thresholds coincide with 
the one in [6], [8]. Any such splitting of a dictionary will yield 
improved sparsity thresholds. However, finding the optimal split, 
for a given dictionary, in terms of obtaining the best possible 
thresholds is a combinatorial problem. 

Our threshold for uniqueness of the solution of (P0) is based 
on a generalization of the uncertainty relation for the two- 
ONB setting [7] to dictionaries in T>(d,a,b). The derivation 
of the threshold for the uniqueness of the solution of (PI) is a 
generalization of the corresponding derivation for the two-ONB 
case reported in [10]. 

Notation: Throughout the paper, we use lowercase boldface 
letters for column vectors, e.g., x, and uppercase boldface letters 
for matrices, e.g., D. For a given matrix D, we denote its 

1 Without loss of generality, we assume a < b in the remainder of the paper. 



conjugate transpose by T) H , and the element in its ith row and 
jth column by [D]jj; furthermore, stands for its zth column, 
and kcrn(D) is its null space. The zth element of a vector x 
is denoted by Xj . The Euclidean or ^-norm of the vector x is 
||x||2 = Vx H x, the £i-norm is defined as ||x||i = X^W' 
and Hxlloo = maxilx.il is the ^-norm of x. The smallest eigen- 
value of the positive-definite matrix G is denoted by A m i n (G). 
For 11 £ K, we define [u] + — max{0, u}. 

II. Previous Results 

In this section, we review relevant sparsity thresholds [5]— [10]. 
The spark of a dictionary D, defined as the cardinality of the 
smallest set of linearly dependent columns of D [6], sheds light 
on the conditions for uniqueness of the solution of (P0). In partic- 
ular, the following result holds: for a given dictionary D, a vec- 
is the unique solution of (P0) if and only if [6], [8] 

||x|| < spark(D)/2 



to 



■0* 



where spark(D) denotes the spark of D. Determining the spark 
of a dictionary is a combinatorial problem and therefore computa- 
tionally expensive, in general. A common approach to overcome 
this problem is to derive lower bounds on spark(D) that are 
explicit in an easy-to-compute parameter of the dictionary. Let, 
for example, the coherence of a dictionary D be defined a^] 



d = max|d, ff d 

i& 1 



(1) 



and let T) gen (d) be the set of all dictionaries with coherence d. 
It is shown in [6], [8], [9] that for any dictionary D in V gen (d) 
we have tha0 

spark(D) > 1 + l/d. (2) 

Consequently, for a dictionary with coherence d, a sufficient 
condition for a vector x to be the unique solution of (P0) is 



No < 



l 



l 



(3) 



It is furthermore shown in [6], [8] that when (|3} holds, the 
solution of (PI) is unique as well (and hence equal to the unique 
solution of (P0)). 

The threshold on the right-hand side (RHS) of ^ applies to 
any dictionary in T) gen (d). This set is, however, rather large and 
the dictionaries within the set can differ widely in their structure: 
from equiangular tight frames [11], where each pair of elements 
in the dictionary achieves the maximum in fll, to dictionaries 
where the maximum is achieved by one pair only. For a given 
dictionary, the threshold on the RHS of ([3]) might therefore be 
rather pessimistic. 

Better sparsity thresholds can be obtained if structural proper- 
ties of the dictionary are explicitly taken into account. This leads 
to sparsity thresholds that apply to smaller sets of dictionaries. 
Let, for example, T> on b(d) C T> gen (d) be the set of dictionaries 
with coherence d that consist of the union of two ONBs. For 

2 In the remainder of the paper, whenever we speak of a vector x, we mean 
that this vector is consistent with the observation y, i.e., it satisfies y = Dx. 
3 For dictionaries consisting of only one element we set d = 0. 
4 Note that d < 1 as we assumed that ||dj||2 = 1, i = 1, . . . , iV. 



any dictionary in V on \,(d), a sufficient condition for x to be the 
unique solution of (P0) is [7], [8], [10] 



|x|| < l/d. 



(4) 



A sufficient condition for x to be the unique solution of (PI) 
(which hence equals the unique solution of (P0)) is [7], [8], [10] 

||x|| < (V2-0.5)/d. (5) 

For small d, the thresholds in Q and |5]) are almost a factor of 2 
larger than the threshold on the RHS of Q. Surprisingly, the 
threshold in <[5j drops below that in Q for d > 2(v / 2 - 1). 

III. Two Novel Sparsity Thresholds 

We focus on the class T>(d, a, b) of dictionaries with coher- 
ence d that consist of two sub-dictionaries with coherence a 
and b, respectively. A generic dictionary in T>(d, a, b) is denoted 
as D = [A B], where A has coherence a and B has coherence b. 
The sub-dictionaries A and B need not be ONBs, need not have 
the same number of elements, and do not need to span C M . Note 
that T>(d, a, b) C V gen (d); hence, the sparsity threshold in ^ 
applies to anydictionary in T>(d, a, b) as well. Furthermore, we 
also have thaj^X> onb (d) C V(d, 0, 0). In Theorem[l]and|2]below, 
we derive sparsity thresholds that apply to the set T>(d, a, b), are 
functions of d, a, and b only, and improve on the threshold that 
would be obtained if one treated the union of two sub-dictionaries 
as a general dictionary with coherence d. The basis for our result 
on the uniqueness of the solution of (P0) is an uncertainty relation 
for dictionaries in T>(d, a, b) stated next. 

Lemma 1: Let D = [A B], D S C MxN , be a dictionary in 
T>(d, a, b). For any vector in C M that can be represented as a 
linear combination of n a linearly independent columns of A and, 
equivalently, as a linear combination of n b linearly independent 
columns of B, the following inequality holds: 

n a n b > [1 - a(n a - 1)]+ [1 - b(n b - / d 2 . (6) 

Proof: See Appendix [A] ■ 
Note that for a — b = 0, |6} reduces to n a n\, > l/d 2 , which is 
the well-known uncertainty relation for the union of two ONBs 
reported in [7, Th. 1]. 

We next provide a lower bound on the spark of dictionaries 
in the set T>(d, a, b). 

Theorem 1: The spark of any dictionary D in the 
set T)(d, a, b) is lower-bounded according to 



where 

and 
with 



spark(D) > x + f(x) 

(1 + a)(l + b) - xb(l + a) 
x(d 2 - ab) + a(l + b) 

x = minjx;,, x s } 



x b = {l + b)/{b+d 2 ) 



5 T>(d, 0, 0), for example, also contains dictionaries that consist of an ONB 
and an orthonormal set which does not form a basis. 



threshold for V onb {d = 0.1)[eq. (4)] 

-threshold for V(d = 0.1, a = b,b)[eq. (7)] 




Figure 1. Sparsity thresholds for dictionaries in T>u en (d), ~D on \>(d), and 
T>(d, a, b). The case d = 0.1 and a = b is considered. Note that when a = b, 
the bound jTJ reduces to ||xj|o < (1 + b)/(d + b). 



and 



dy/(l + a)(l + 6) - a-ab 
d 2 - a& ' 



if a 



otherwise. 



Proof: See Appendix [B] ■ 
Theorem[T]yields the following sufficient condition for a vector x 
to be the unique solution of (P0) for any D <G T)(d, a, b): 

||x|| < [x + f(x)]/2. (7) 

The threshold in |7]l reduces to the one in Q when a = b = d, 
or when x\, < x s and b — d. In all other cases, the threshold 
in |7]l is strictly larger than that in ([3J, as shown in Appendix [C] 
Furthermore, (|7]i reduces to Q when a = b = 0. As illustrated 
in Fig.[T]for the case a = b, the improvement of the threshold 
in |7} over that in (|3} is significant when a,b d. 

We next present a sparsity threshold that guarantees the unique- 
ness of the solution of (PI) (which is hence equal to the unique 
solution of (P0)) for any dictionary in T>(d, a, b). 

Theorem 2: For any dictionary D in T>(d, a, b) a sufficient 
condition for (PI) to have a unique solution is 



■5[e- (d + 36)] 



2(d 2 - 
+ 2d 2 



b 2 ) ' 
-3b-d5 



if b < d and n(d, b) > 1, 



2(d 2 + b) 



otherwise 



(8) 



with 



n(d, b) = 



5y/2d(b + M+e) - 2d- 2b(8 + d) 
2(d 2 - b 2 ) 



where 6 = 1 + b and e = 2y/2 y / d(b + d). 

Proof: See Appendix [D] ■ 
The threshold in (|8]l depends on b and ei only. An improved 
threshold, which depends on a as well, can be obtained through 
minor modifications of the proof in Appendix [D] The resulting 
expression is, however, unwieldy. 

The threshold in ([H} reduces to the one in ([3]) whenever b = d 
(independently of the value of a). Otherwise, the threshold in ([8]) 
is strictly larger than that in ([3]). Hence, viewing a given dictio- 



nary D as the concatenation of two sub-dictionaries A and B 
(i.e., splitting the dictionary D into A and B) yields improved 
sparsity thresholds, as a function of d, a, and b, compared to the 
threshold (1 + l/d)/2 in [6], [8]. 

For a = b = 0, the threshold in ([8]) reduces to 



Wo< 



(y/2-0.5)/d, 

l + (l-d)/(2d 2 ), 



if d < l/y/2, 



otherwise. 



(9) 



Note that d9| coincides with the two-ONB threshold in Q when 
d < l/v%but improves on |5]l for d > \j\[2. Furthermore, 
differently from the threshold in the threshold in |9]) cannot 
fall below the threshold for T> gen (d) given by (1 + l/d)/2. 

Appendix A 
Proof of LemmaQ] 

Let y be a vector in C M that can be represented as a linear 
combination of n a linearly independent columns of A and, 
equivalently, as a linear combination of rib linearly independent 
columns of B. Let V and Q be the sets containing the indices 
of the participating columns in A and B, respectively. Then 

y = YsievPi** = 2~2jeQ <lj h j for some scalars {pj and {^}. 
Let p be the n a -dimensional vector containing the scalars {pi}, 
and, similarly, let q be the rib-dimensional vector containing the 
scalars {qj}- The following chain of inequalities holds: 

llylla = EE^afb^ < £ £>| |^| |af b,| 

iev jeQ iev jeQ 

< d||p||i||q||i < dVr^||p|| 2A M||q|| 2 . (10) 

Denote by A-p the matrix containing the columns of A corre- 
sponding to the index set V. Furthermore, let Gp = ApAp. 
Then, we can lower-bound the ^2-norm of y as follows: 

in (Gp)||p||2. (11) 




Using analogous steps (and notation), we also get 

||y||l = q ff BgB Q q > A min (G Q )||q|||. (12) 
= G Q 

Note that both A m ; n (G-p) and A m i n (Gg) are strictly positive, 
because the columns of A-p and of Bg are linearly independent, 
by assumption. Inserting (jTTJ and ([12} into ( fT0] > yields 



\Jn a n h > - J\ 



i 

(Gp)A min (Gg). 



(13) 



Next, proceeding as in [9, Lem. 2.3], we bound A m ; n (G73) 
and A m i n (Gq) from below using Gersgorin's disc theorem [12, 
Th. 6.1.1] and obtain A m in(G-p) > max{0, 1 — a(n a — 1)} and 
Amin(Gq) > max{0, 1 — &(ri(, — 1)}, respectively. The proof 
is concluded by inserting these two lower bounds into ( fT3] >. 

Appendix B 
Proof of TheoremQ] 

Let D be an arbitrary dictionary in T>(d, a, b), and let Z(D) be 
the cardinality of the smallest set of linearly dependent columns 
of D consisting of n a > 1 linearly independent columns of A 



and n b > 1 linearly independent columns of B. By the definition 
of the spark [6, Def. 1], we have that 

spark(D) = min{spark(A) , spark(B), Z(D)}. (14) 
We next lower-bound each term in ( p~4] >. From |2} we get 

spark(A) > 1 + 1/a and spark(B) > 1 + 1/6. 

Since a < b, by assumption, we obtain 

spark(D) > min{l + 1/6, Z(D)}. (15) 

We now seek a lower bound on the RHS of (H3}. To this end, we 
start by noting that Z(D) = n a + n b . In the light of ([15}, we 
need to focus only on dictionaries that have n a + n b < 1 + 1 jb. 
From the uncertainty relation in Lemma [T] we know that the 
product n a rib cannot be arbitrarily small. More specifically, we 
have that n a rib > [1 — a(n a — 1)][1 — b(rib — l)]/d 2 . Solving 
for n a , we ge|^] 

^ (l + a)(l + b)-n b b(l + a) 
nb{d i — ab) + a(l + b) 
Finally, adding rib on both sides yields 

Z(D) = n„ + n b > f(n b ) + n b . (16) 

To obtain an expression that is explicit in d, a, and b only, we 
further lower-bound the RHS of ( fTo*) by minimizing f (rib) + rib 
over rib, under the additional constraints n a > 1 and n b > I 
(implied by assumption). The resulting bound reads 

Z(D) > min [rift + max{/(rib), 1}]. 



Since 



mm 



i[n & + max{/(n fc ),l}] < [n b + max{f(n b ), 1}] \ nb=1/h 



= 1 + 1/6 

we can lower-bound the RHS of ( fT4| ) by 

spark(D) > min [rib + niax{/(ri{,), 1}] 

n b >l 

> min [x + max{/(i), 1}]. 

X>1 



(17) 



Here, in the last step, we replaced the integer parameter n b 
by the real-valued parameter x. We next compute the mini- 
mum in (IT) . The function f(x) is monotonically decreasing. 
Furthermore, the equation f(x) — 1 has the unique solu- 
tion Xb = (1 + 6)/(6 + d 2 ) > 1, where the inequality follows 
because rf < 1 by definition. We can, therefore, rewrite ( fT7] i as 

min[x + max{/(x), 1}] = min [x + f(x)]. (18) 

In the case a = b = d, the function g(x) = x + f(x) reduces to 
the constant 1 + l/d so that spark(D) > 1 + 1/d. In all other 
cases, the function 5(2:) is strictly convex for a; > 0. Furthermore, 
we have g(l) > g(xb) as a consequence of the assumption a < b. 
Hence, the minimum in ( fT8] i is achieved — as a result of the 
convexity of g(x) — either at the boundary point Xb, or at the 



Equivalently, we could solve for nj, and obtain the same final result. 



> 1. 



(19) 



stationary point x s of <?(ir), which is given by 

_ ^(1 + a)(l + 6) - a - ab 
Xs= a^a~b 

The inequality in ( fT9] > follows from the convexity of g{x) and 
the fact that g(l) > g(x b )- 

Appendix C 

The case a = b = d: The threshold in |7]i reduces to the 
well-known [6], [8] threshold (1 + l/d)/2 in |3j. 
The case Xb < x s : The threshold in |7| equals 



■ + f{x) x b + f{x b ) 



2 2 
It is now easily verified that 

2 1 b+d 2 



1 



1 



> 



1 



b+d 2 

d. 



for all 6 < d < 1 with equality only if 6 = 

The case Xb > x s : Set A = y / (T + a)(l + 6) — d. The 
function x s +f(x s ), which depends on the variables d, a, and 6, is 
strictly monotonically decreasing in a as long as bjd < A < d/b. 
The inequality A < d/b is always satisfied, because a < b < d. 
The inequality b/d < A holds whenever x s < which is 
satisfied by assumption. Hence, we have that 



fix.) 



> 



fix.) 



1 



a—b 



Here, equality holds only for the case a - 
separately above. 



6 If 1 
-6^2 ( 1+ d 



b = d, already treated 



Appendix D 
Proof of Theorem|2] 

The proof follows closely that of [10, Prop. 3], which deals 
with the case where A and B are ONBs. Our proof makes use 
of the following result. 

Proposition 1 ( [8, hem. 1, Lem. 2], [10, Prop. 2]): 
Let Sl(v) denote the sum of the largest (in absolute 
value) L entries of the vector v s and assume that 

Pl(D)= max 

v^O, v e kern(D) ||v||i 2 

Then, for all x G C N with ||x||o < L and for all nonzero v G 
kern(D) we have that ||x + v||o > ||x||o and ||x + v||i > ||x||i. 

Proposition[T]implies that x is the unique solution of both (P0) 
and (PI). We will use this proposition as follows: First, we will 
derive an upper bound on Pl(D) that is explicit in d, b, and L. 
Second, we will impose that this upper bound falls below 1/2, 
and then solve for L to get (|8). 

Let v be an arbitrary nonzero vector in kern(D). Be- 
cause D = [A B], it is convenient to view v as a concatenation 
of two vectors w and z that satisfy Aw = — Bz. Let n be the 
number of elements in w that belong to the set of L > 1 largest 
(in absolute value) entries of v and let W be the corresponding 
index set. Similarly, let Z be the index set corresponding to the 
L — n elements of z that belong to the set of largest (in absolute 



value) entries of v. Two situations — that we treat separately — 
may arise: n > L/2 and n < L/2. 

The case n > L/2: We assume, without loss of generality, 
liew\ w i\ = L Furthermore, we set EigwKI = 7> so 
I+7. Then, we can write Sx(v)/[jv||i as follows: 



thaf]£ 
that] 



w 1 



Sl(v) E 



tewi 



E 



i£Z\ 



i + E 



IMIi llwlli + HzlU 



I + 7+INI1 



(20) 



l + 7+||z||i 

We next derive an upper bound on UzHoo and a lower bound 
on ||z||i both of which are explicit in 6 and d. The vectors w 
and z satisfy the following equality 



B fl Aw 



B H Bz. 



(21) 



To get an upper bound on ||z||oo> we next bound the ^-norm 
of both sides of d2T|i. For the LHS we have 



lIB^AwHoo < maxIp^A], JlMd < d||w|| x . (22) 



1,3 



The RHS of ( f2T| can be lower-bounded as follows. Let G 



B H B and note that, by assumption, [G]i 
for i ^ j. Then 



|Gz|U = 



z i + / 



> max 



1 and [G]j,J < 6 



= (l + ftJiHU-dHi. 

Combining ( |22| > and ( |23| l, we get 

(1 + 6)||z||oo - < dllwll! 
or, equivalently, 

d||w||i + 6||z||i d(l + 7) + 6||z|| 



< 



(1 + 6) (1 + 6) 

Since the vectors w and z also satisfy 

A fl Aw = A ff Bz 

steps similar to the ones used to arrive at ( p4| > yield 

(1 + oJHwlloo - o||w||i < d||z||i. 



(23) 



(24) 



(25) 



(26) 



Invoking the assumption a < 6, we can lower-bound the LHS 
of l|2^ according to (1 + a)||w|| oc> — a||w||i > (1 + 6)||w|| 00 — 
6j|w||i, which, upon insertion into |26|, yields 

(l + 6)||w|| 



&Hw|U > (l + 6)/n-6(l+ 7 ) 



(27) 

In the last step, we used HwHoo > 1 /n, as a consequence of the 
normalization Eiewl -1 ^! = 1 ( reca ll that W has cardinality n). 
Using ([25]) and ((27]) in |20], we obtain 

S L (y) < 1 + (L - n)(d - 6)(1 + 7 )/(l + 6) , (i - n)6 



||v||i I + 7+ [(l + 6)/n- 6(1 + 7)] /d 
= h(n,j,L). 



(28) 



This upper bound can be further simplified if we re- 
place h(n, 7, L) by its maximum with respect to n and 7. 
The function h(n,j,L) is monotonically decreasing in 7 for 
n > L/2. Hence, we get 

max 6(n,7,L)< max h(n, 0, L) < max 0, L). 

L/2<n<L L/2<n<L x>l 

For b = d, the function /i(x, 0, L) does not depend on x and 
reduces to 

%,0,i) = Ld/(l + d). 

Thus, for 6 = d, the inequality h(x,0,L) < 1/2 is satisfied 
(independently of x) if and only if L < (1 + l/d)/2, which 
coincides with the threshold previously found for general dictio- 
naries in [6], [8]. 

For 6 < d and fixed L, the function h(x, 0, L) is concave 
in x for x > 0. Furthermore, we have that 6,(1,0, L) > 
limj-^oo 6(2;, 0, L). Denoting the stationary point of h(x, 0, L) 
byx s — x s (d, b, L), the properties of h(x, 0, L) just stated imply 
that 



max6(x, 0, L) 

X>1 



h(x s ,0,L), 
6(1,0,L), 



if S a (d, b,L) > 1 
otherwise. 



If we now impose that max 3: > 1 h(x,0,L) < 1/2 and solve 
for i, we get the final result (jHJ. The condition «(d, 6) > 1 
in (|8]l is obtained by solving h(x s ,0, L) — 1/2 and inserting the 
resulting value of L into the condition x s (d, b,L) > 1. 

The case n < L/2: In this case, we repeat the steps detailed 
above for the case n > L/2, swapping the roles of w and z, and 
obtain an expression analogous to (|28]). The resulting threshold 
coincides with ([8]). 
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A scaling of v does not alter the ratio £x(v)/|| v|j 1. 



